HDDS-10483. Container Balancer should only move containers with size greater than 0 bytes #6447
Conversation
…greater than 0 bytes
sumitagrawl
left a comment
sumitagrawl
left a comment
There was a problem hiding this comment.
@sarvekshayr Thanks for working over this, have minor comment
...m/src/test/java/org/apache/hadoop/hdds/scm/container/balancer/TestContainerBalancerTask.java
Outdated
Show resolved
Hide resolved
|
Why is this change needed? If there is a skew in zero size objects it will show in the heartbeat load form a Datanode. |
@kerneltime |
...server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/FindSourceGreedy.java
Outdated
Show resolved
Hide resolved
siddhantsangwan
left a comment
siddhantsangwan
left a comment
There was a problem hiding this comment.
@sarvekshayr Thanks for working on this. The code change looks good, but we should have an explicit unit test that asserts balancer only selects positive size containers. This ensures we don't have to depend on another unit test that's meant to test a different behaviour.
Added a unit test to assert the expected behaviour of the balancer. |
|
@sarvekshayr can you look into the CI failures? |
|
The balancer test had probably failed because of #6481. Recon failure looks unrelated. |
|
Merged to master with green CI. Thanks @sarvekshayr and all the reviewers. |
…greater than 0 bytes (apache#6447)
…greater than 0 bytes (apache#6447) (cherry picked from commit 80c2311)
…s with size greater than 0 bytes (apache#6447) (apache#112) Co-authored-by: Sarveksha Yeshavantha Raju <79865743+sarvekshayr@users.noreply.github.com>
…greater than 0 bytes (apache#6447) (cherry picked from commit 80c2311)
5 participants
What changes were proposed in this pull request?
Introduced a check on the size containers allowed to move during the balancing process.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-10483
How was this patch tested?
Added a unit test
balancerShouldMoveOnlyPositiveSizeContainersto assert that balancer selects only positive size containers.Also tested using the existing test
balancerShouldObeyMaxSizeLeavingSourceLimit()inTestContainerBalancerTask.java. It passed successfully, ensuring that the balancer selected only containers with a size greater than 0 bytes, even when zero-byte containers were present.